Multimodal Engagement Prediction in Multiperson Human–Robot Interaction

The ability to measure the engagement level of humans interacting with robots paves way towards intuitive and safe human-robot interaction. Recent approaches achieve reasonable progress in predicting human physically situated environments. However, estimation is still a challenging problem especially an open-world environment due difficulty creating monitoring variety social cues real-time. Furthermore, interactions may involve group subjects simultaneously robot, which increases prediction complexity. In this paper, we design real-time system for generalization capability. We propose estimate using three-stage approach based on combination learning-based rule-based approaches. Firstly, state-of-the-art deep learning methods are used extract features from input frames. Then, simple neural network focus attention score by incorporating gaze head pose assigning all scene face recognition algorithm. Finally, classification predict state subject initiate/terminate interaction robot. To effectively evaluate our system, access each phase separately. Additionally, use online evaluation study allowed interact freely industrial Our model achieves average 96%, 90%, 93% precision, recall, F-score respectively.